In so-called second-screen applications, a viewer is typically watching a streamed video on a primary or main device and watching (or listening to) related content on a second device. In a common situation a first receiver unit associated with the primary screen receives a multicast transmission (that is, a transmission made to many end-users at substantially the same time), and a separate unicast (one to one) transmission is sent to a second receiver unit, typically a handheld tablet or smartphone device. Examples of such related content may be alternative camera angles, different soundtracks (e.g. language), play-along games, etc. These are usually of more specialized interest, and/or are interactive and therefore bespoke to the end-user, and thus are only transmitted on demand.
Other examples of multiple views include the creation of an immersive panoramic or full-360 degree view made by “stitching” several camera views together.
In many cases, the nature of the related content requires precise synchronization between the primary device and the second device (alternative camera angles, subtitles, stitching of views, multiple viewers, etc.). This can be difficult to achieve. In particular, the multicast transmission to a primary device may be over a wired or cabled broadband connection whereas a second, unicast, stream is typically transmitted over a wireless connection with a narrower bandwidth, either independently or relayed from the primary device over a local wireless connection. Transmission and buffering delays are unlikely to be identical in the two paths.
The displays to be synchronized may be of video content, but other display types are also possible, such as text (e.g. subtitles) or audio content.
Another situation in which synchronization can be important is when multiple viewers are linked to a single streaming source, and are linked by a “social chat” facility. It is desirable in such a case that all viewers are watching the same content in synchronization.
Where such transmissions are of live streams, one solution is to include an additional delay to allow adjustment of the start times of the streams to allow their synchronization. This is normally achieved by reference to an absolute time or clock, as described for example in ETSI DVB technical specifications. For example, the MPEG-DASH (Dynamic Adaptive Streaming over HTTP) protocol has segment header data including an absolute time reference enabling video synchronization based on those absolute time references. However some streaming protocols lack such an absolute reference. An example is the HLS (HTTP Live Streaming) protocol, which is a proprietary unicast streaming protocol for delivery of live or on-demand media, and is currently widely used for streaming video on mobile devices.
A further problem arises if the user wishes to interrupt the playing of the stream, and play a section out of sequence—for example to replay a section or, if the stream is not being viewed live, to “fast forward” to a point later in the stream. A user uses a “seek” command in the viewing controls to identify from which point in the stream the user wishes to resume playing. Various “trick play” systems are available to allow the user to identify the relevant part of the video stream in question and, having done so, the “seek” command then instructs the viewing apparatus to play from the segment in question. In such circumstances it is then necessary to re-synchronize the stream played on the second screen. This requires a second “seek” command, to identify the correct segment in the second stream and to resume playing it at the right time. It would be inconvenient and distracting for a user to have to repeat the “replay/fast forward” process for the second stream, and difficult to synchronize them exactly by eye.